Systems and methods for analyzing perfusion-weighted medical imaging using deep neural networks

ABSTRACT

Systems and methods for analyzing perfusion-weighted medical imaging using deep neural networks are provided. In some aspects, a method includes receiving perfusion-weighted imaging data acquired from a subject using a magnetic resonance (“MR”) imaging system and modeling at least one voxel associated with the perfusion-weighted imaging data using a four-dimensional (“4D”) convolutional neural network. The method also includes extracting spatio-temporal features for each modeled voxel and estimating at least one perfusion parameter for each modeled voxel based on the extracted spatio-temporal features. The method further includes generating a report using the at least one perfusion parameter indicating perfusion in the subject.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application represents the national stage entry of InternationalApplication PCT/US2016/066877 filed Dec. 15, 2016, which claims benefitof U.S. Provisional Application 62/267,328 filed Dec. 15, 2015, which isincorporated herein by reference in its entirety for all purposes.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with government support under NS076534, awardedby the National Institutes of Health. The government has certain rightsin the invention.

BACKGROUND

The field of the invention is directed to magnetic resonance (“MR”)imaging. More particularly, the present disclosure is directed tosystems and methods for performing and analyzing perfusion-weightedmedical imaging.

Any nucleus that possesses a magnetic moment attempts to align itselfwith the direction of the magnetic field in which it is located. Indoing so, however, the nucleus precesses around this direction at acharacteristic angular frequency (Larmor frequency), which is dependenton the strength of the magnetic field and on the properties of thespecific nuclear species (the magnetogyric constant γ of the nucleus).Nuclei which exhibit these phenomena are referred to herein as “spins.”

When a substance such as human tissue is subjected to a uniform magneticfield (polarizing field B₀), the individual magnetic moments of thespins in the tissue attempt to align with this polarizing field, butprecess about it in random order at their characteristic Larmorfrequency. A net magnetic moment M_(Z) is produced in the direction ofthe polarizing field, but the randomly oriented magnetic components inthe perpendicular, or transverse, plane (x-y plane) cancel one another.If, however, the substance, or tissue, is subjected to a transientelectromagnetic pulse (excitation field B₁) which is in the x-y planeand which is near the Larmor frequency, the net aligned moment, M_(Z),may be rotated, or “tipped”, into the x-y plane to produce a nettransverse magnetic moment Mt, which is rotating, or spinning, in thex-y plane at the Larmor frequency. The practical value of thisphenomenon resides on signals that are emitted by the excited spinsafter the pulsed excitation signal B₁ is terminated. Depending uponchemically and biologically determined variable parameters such asproton density, longitudinal relaxation time (“T1”) describing therecovery of M_(Z) along the polarizing field, and transverse relaxationtime (“T2”) describing the decay of M_(t) in the x-y plane, this nuclearmagnetic resonance (“NMR”) phenomena is exploited to obtain imagecontrast and concentrations of chemical entities or metabolites usingdifferent measurement sequences and by changing imaging parameters.

When utilizing NMR to produce images and chemical spectra, a techniqueis employed to obtain NMR signals from specific locations in thesubject. Typically, the region to be imaged (region of interest) isscanned using a sequence of NMR measurement cycles that vary accordingto the particular localization method being used. To perform such ascan, it is, of course, necessary to elicit NMR signals from specificlocations in the subject. This is accomplished by employing magneticfields (G_(x), G_(y), and G_(z)) which have the same direction as thepolarizing field B₀, but which have a gradient along the respective x, yand z axes. By controlling the strength of these gradients during eachNMR cycle, the spatial distribution of spin excitation can be controlledand the location of the resulting NMR signals can be identified from theLarmor frequencies typical of the local field. The acquisition of theNMR signals is referred to as sampling k-space, and a scan is completedwhen sufficient NMR cycles are performed to fully or partially samplek-space. The resulting set of received NMR signals are digitized andprocessed to reconstruct the image using various reconstructiontechniques.

To generate an MR anatomic image, gradient pulses are typically appliedalong the x, y and z-axis directions to localize the spins along thethree spatial dimensions, and MR signals are acquired in the presence ofone or more readout gradient pulses. An image depicting the spatialdistribution of a particular nucleus in a region of interest of theobject is then generated, using various post-processing techniques.Typically, the hydrogen nucleus (1H) is imaged, though otherMR-detectable nuclei may also be used to generate images.

Perfusion-weighted MRI is a common imaging technique that is used in theclinical treatment of patients with brain pathologies, such as stroke orcancer. Perfusion-weighted images (“PWI”) are typically obtained byinjecting a contrast bolus, such as a gadolinium chelate, into apatient's bloodstream and imaging as the bolus passes through thepatient using dynamic susceptibility contrast (“DSC”) or dynamiccontrast enhanced (“DCE”) imaging techniques. The susceptibility effectof the paramagnetic contrast leads to signal loss that can be used totrack contrast concentration in specific tissues over time. By applyingvarious models to the resulting concentration-time curves, a number ofperfusion parameters can be determined, such as blood volume (“BV”),blood flow (“BF”), mean transit time (“MTT”), time-to-peak (“TTP”),time-to-maximum (“T_(max)”), maximum signal reduction (“MSR”), firstmoment (“FM”), and others.

Previously, PWI have been analyzed by using deconvolution methods. Inthese approaches, the measured concentration of contrast in each voxelas a function of time is expressed as the convolution between anarterial input/venous output concentration and a residual curve, whichexpresses the residual amount of contrast in each voxel. By applyingvarious deconvolution algorithms, such as algorithms based on singlevalue decomposition (“SVD”), different perfusion parameters may beobtained and used to determine a chronic or acute condition of thepatient. For example, T_(max) and MTT have been used to predict a riskof infarction.

However, there are growing concerns that perfusion parameters obtainedusing deconvolution techniques are less predictive due to errors anddistortions introduced during the deconvolution process. In addition,values for the generated parameters and hence conclusions drawnthereupon can vary depending upon the specific models and modelassumptions utilized. For example, the infarct core and penumbra regionsor their ratio, computed using TTP or T_(max), have been used to guidetreatment decisions in acute stroke patients. However, the appropriatethresholds for TTP or T_(max) identifying core and penumbra regions arestill subject to debate.

In light of the above, there is a need for improved image analysistechniques that can provide accurate information for the diagnosis andtreatment of patients, such as information regarding the irreversiblyinfarcted tissue or abnormal tissue.

SUMMARY

The present disclosure overcomes the drawbacks of previous technologiesby providing systems and methods for analyzing perfusion-weightedmedical imaging using a neural network architecture. In particular, aconvolutional neural network (“CNN”) architecture may be used. Aspectsof this technology include building and applying deep CNNs withfour-dimensional (“4D”) convolutions to extract imaging features fromperfusion-weighted medical imaging. The imaging features can be used togenerate information useful in diagnosis and treatment of variouspatient conditions.

In accordance with one aspect of the disclosure, a method for analyzingperfusion-weighted medical imaging using deep neural networks isprovided. The method includes receiving perfusion-weighted imaging dataacquired from a subject using a magnetic resonance (“MR”) imaging systemand modeling at least one voxel associated with the perfusion-weightedimaging data using a four-dimensional (“4D”) convolutional neuralnetwork. The method also includes extracting spatio-temporal featuresfor each modeled voxel and estimating at least one perfusion parameterfor each modeled voxel based on the extracted spatio-temporal features.The method further includes generating a report using the at least oneperfusion parameter indicating perfusion in the subject.

In accordance with another aspect of the disclosure, a system foranalyzing perfusion-weighted imaging. The system includes an inputconfigured to receive perfusion-weighted imaging data acquired from asubject using a magnetic resonance (“MR”) imaging system and a processorprogrammed to execute instructions stored in a non-transitory computerreadable media to access the perfusion-weighted imaging data. Theprocessor is also programmed to generate an input patch for eachselected voxel associated with the perfusion-weighed imaging data, andapply a four-dimensional (“4D”) convolutional neural network to thegenerated input patch to extract spatio-temporal features. The processoris further programmed to compute at least one perfusion parameter usingthe spatio-temporal features extracted, and generate a report using theat least one perfusion parameter. The system further include an outputfor displaying the report to communicate perfusion in the subject.

The foregoing and other aspects and advantages of the invention willappear from the following description. In the description, reference ismade to the accompanying drawings that form a part hereof, and in whichthere is shown by way of illustration a preferred embodiment of theinvention. Such embodiment does not necessarily represent the full scopeof the invention, however, and reference is made therefore to the claimsand herein for interpreting the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will hereafter be described with reference to theaccompanying drawings, wherein like reference numerals denote likeelements. The patent or application file contains at least one drawingexecuted in color. Copies of this patent or patent applicationpublication with color drawing(s) will be provided by the Office uponrequest and payment of the necessary fee.

FIG. 1 is a schematic diagram of an example system, in accordance withaspects of the present disclosure.

FIG. 2 is a flowchart setting forth steps of a process, in accordancewith aspects of the present disclosure.

FIG. 3 is an illustration of a four-dimensional (“4D”) feature detector,in accordance with aspects of the present disclosure.

FIG. 4 is an illustration showing a 4D convolutional neural networkarchitecture, in accordance with aspects of the present disclosure.

FIG. 5A shows a comparison between a FLAIR image (top) and a probabilityof infarction map (bottom), generated in accordance with aspects of thepresent disclosure.

FIG. 5B shows a comparison between a labeled perfusion map (top) and aperfusion map (bottom) estimated in accordance with the presentdisclosure.

FIG. 6 shows example spatio-temporal filters learned in accordance withaspects of the present disclosure.

FIG. 7 is a block diagram of an example magnetic resonance imaging(“MRI”) system, in accordance with aspects of the present disclosure.

DETAILED DESCRIPTION

The present disclosure is directed to systems and methods for analyzingtemporal medical imaging data, and more specifically perfusion-weightedmedical imaging data. Current techniques apply singular valuedecomposition algorithms to deconvolve image signals to obtain perfusionparameters, such as time-to-maximum (“T_(max)”). These parameters arethen used to characterize tissue properties, and predict tissue andpatient outcomes. However, studies have found that the deconvolutionprocess can introduce distortions that influence the measurement ofperfusion parameters, such as T_(max), and the decoupling of delay maynegatively impact prediction. In addition, perfusion parameters can varywhen different computational methods are used, leading to aninconsistency in prediction.

To address the drawbacks of current approaches, the present disclosureintroduces a generalized deep convolutional neural network (“CNN”)architecture that can estimate perfusion parameters, as well asautomatically learn hierarchical spatio-temporal features. Thesefeatures allow for a more precise quantification of perfusion-weighedimaging, and can support a variety of biomedical applications, includingpredicting tissue death (infarction) at the voxel level in acute strokepatients. Estimating perfusion parameters accurately and predictingtissue death provides valuable information that may help physiciansdecide between competing treatment options.

An important advantage of present technology includes a four-dimensional(“4D”) CNN architecture that can be used to capture spatial informationto model image voxels in three dimensions (“3D”), with a fourthdimension capturing the temporal information of each voxel acquiredduring perfusion imaging. This enables the deep CNN's to learn 4Dspatio-temporal filters (rather than 3D filters) that can extracthierarchical spatio-temporal features, which are more accurate thantraditional variables, such as T_(max), in predicting tissue outcome. Inaddition, the present CNN architecture may be used to learn hiddenfeatures that can better estimate the perfusion parameters without needfor performing deconvolution analysis used in prior technologies.

Turning now to FIG. 1, a block diagram of an example system 100, inaccordance with aspects of the present disclosure, is shown. In general,the system 100 may include an input 102, a processor 104, a memory 106,and an output 108, and may be configured to carry out steps analyzingperfusion-weighted imaging in accordance with aspects of the presentdisclosure.

As shown in FIG. 1, the system 100 may communicate with one or moreimaging system 110, storage servers 112, or databases 114, by way of awired or wireless connection. In general, the system 100 may be anydevice, apparatus or system configured for carrying out instructionsfor, and may operate as part of, or in collaboration with variouscomputers, systems, devices, machines, mainframes, networks or servers.In some aspects, the system 100 may be a portable or mobile device, suchas a cellular or smartphone, laptop, tablet, and the like. In thisregard, the system 100 may be a system that is designed to integrate avariety of software and hardware capabilities and functionalities, andmay be capable of operating autonomously. In addition, although shown asseparate from the imaging system 110, in some aspects, the system 100,or portions thereof, may be part of, or incorporated into, the imagingsystem 100, such as the magnetic resonance imaging (“MRI”) systemdescribed with reference to FIG. 8.

Specifically, the input 102 may include different input elements, suchas a mouse, keyboard, touchpad, touch screen, buttons, and the like, forreceiving various selections and operational instructions from a user.The input 102 may also include various drives and receptacles, such asflash-drives, USB drives, CD/DVD drives, and other computer-readablemedium receptacles, for receiving various data and information. To thisend, input 102 may also include various communication ports and modules,such as Ethernet, Bluetooth, or WiFi, for exchanging data andinformation with these, and other external computers, systems, devices,machines, mainframes, servers or networks.

In addition to being configured to carry out various steps for operatingthe system 100, the processor 104 may also be programmed to analyzeperfusion-weighted imaging according to methods described herein.Specifically, the processor 104 may be configured to executeinstructions, stored in a non-transitory computer readable-media 116.Although the non-transitory computer readable-media 116 is shown in FIG.1 as included in the memory 106, it may be appreciated that instructionsexecutable by the processor 104 may be additionally or alternativelystored in another data storage location having non-transitory computerreadable-media.

In some aspects, the processor 104 may be configured to receive andprocess perfusion-weighted imaging data to generate a variety ofinformation, including perfusion parameters, or parameter maps. By wayof example, perfusion-weighted imaging data may include dynamicsusceptibility contrast (“DSC”) imaging data and dynamic contrastenhanced (“DCE”) imaging data, as well as other data. In some aspects,the processor 104 may access information and data, includingperfusion-weighted imaging data, stored in the imaging system 110,storage server(s) 112, database(s) 114, PACS, or other data storagelocation. In other aspects, the processor 104 may direct acquisition ofperfusion-weighted and other imaging data, for example, using an MRIsystem, as described with reference to FIG. 8. The processor 104 mayalso preprocess the received data and information. For example, theprocessor 104 may generate one or more images using acquired or receiveddata. In addition, the processor 104 may segment certain portions of theimages, such as skull-stripping and ventricles' removal.

In processing perfusion-weighted imaging data, the processor 104 maygenerate an input patch for each selected voxel and apply afour-dimensional (“4D”) convolutional neural network to the generatedinput patch to extract spatio-temporal features. Each input patch may bedefined by a slice number s, a width w, a height h, and a volume numberor time t, where the slice number, width and height represent spatialinformation while the volume number represents temporal information(FIG. 3).

Based on the extracted features, the processor 104 may then compute avariety of information for each selected voxel, including one or moreperfusion parameters perfusions. In processing multiple voxels, theprocessor 104 may generate one or more images or parameter maps. Exampleperfusion parameters or parameter maps include blood volume (“BV”),blood flow (“BF”), mean transit time (“MTT”), time-to-peak (“TTP”),time-to-maximum (“T_(max)”), maximum signal reduction (“MSR”), firstmoment (“FM”), and others.

As mentioned, the 4D convolutional neural network includes spatialinformation and temporal information, and may include an input layer, aconvolutional layer, a pooling layer, a first fully-connected layer or anon-linear layer, a second fully-connected layer, and an output layer.In some aspects, the 4D convolution neural network may include anon-linear layer inserted between each of the first convolutional layerand the pooling layer. For the output layer, the 4D convolutional neuralnetwork may include a classifier layer or non-linear layer connected tothe second fully-connected layer. In some aspects, the processor 104 mayalso be programmed to compute, using the classifier layer, variousprobabilities based on the spatio-temporal features extracted forselected voxels. The processor 104 may then generate various probabilitymaps using the computed probabilities, including a color-coded mapindicating the spatial distribution for the probability of infarction.In other aspects, the processor 104 may be programmed to compute, usingthe non-linear layer, various perfusion parameters based on thespatio-temporal features extracted for selected voxels.

In some aspects, the processor 104 may be configured to identify variousimaged tissues based on computed perfusion parameters. For example, theprocessor 104 may identify infarct core and penumbra regions, as well asregions associated with abnormal perfusion. The processor 104 may befurther programmed to determine a condition of the subject. For example,based on identified tissues or tissue regions, the processor 104 maydetermine a risk to the subject, such as a risk of infarction.

The processor 104 may also be configured to generate a report, in anyform, and provide it via output 108. In some aspects, the report mayinclude various raw or processed maps or images, or color-coded maps orimages. For example, the report may include anatomical images, maps ofCBF, CBV, MTT, TPP, T_(max) and other perfusion parameters as well asprobability maps. In some aspects, the report may indicate specificregions or tissues of interest, as well as other information. The reportmay further indicate a condition of the subject or a risk of the subjectto developing an acute or chronic condition, such as a risk ofinfarction.

Embodiments of the present technology may be described with reference toflowchart illustrations of methods and systems according to embodimentsof the technology, and/or algorithms, formulae, or other computationaldepictions, which may also be implemented as computer program products.In this regard, each block or step of a flowchart, and combinations ofblocks (and/or steps) in a flowchart, algorithm, formula, orcomputational depiction can be implemented by various means, such ashardware, firmware, and/or software including one or more computerprogram instructions embodied in computer-readable program code logic.As will be appreciated, any such computer program instructions may beloaded onto a system 100 as described with reference to FIG. 1, as wellas any other computer, including without limitation a general purposecomputer or special purpose computer, or other programmable processingapparatus to produce a machine, such that the computer programinstructions which execute on the computer or other programmableprocessing apparatus create means for implementing the functionsspecified in the block(s) of the flowchart(s).

Accordingly, blocks of the flowcharts, algorithms, formulae, orcomputational depictions support combinations of means for performingthe specified functions, combinations of steps for performing thespecified functions, and computer program instructions, such as embodiedin computer-readable program code logic means, for performing thespecified functions. It will also be understood that each block of theflowchart illustrations, algorithms, formulae, or computationaldepictions and combinations thereof described herein, can be implementedby special purpose hardware-based computer systems which perform thespecified functions or steps, or combinations of special purposehardware and computer-readable program code logic means.

Furthermore, these computer program instructions, such as embodied incomputer-readable program code logic, may also be stored in acomputer-readable memory in the form of non-transitory computer-readablemedia, that can direct a computer or other programmable processingapparatus to function in a particular manner, such that the instructionsstored in the computer-readable memory produce an article of manufactureincluding instruction means which implement the function specified inthe block(s) of the flowchart(s). The computer program instructions mayalso be loaded onto a computer or other programmable processingapparatus to cause a series of operational steps to be performed on thecomputer or other programmable processing apparatus to produce acomputer-implemented process such that the instructions which execute onthe computer or other programmable processing apparatus provide stepsfor implementing the functions specified in the block(s) of theflowchart(s), algorithm(s), formula(e), or computational depiction(s).

It will further be appreciated that the terms “programming” or “programexecutable” as used herein refer to one or more instructions that can beexecuted by a processor to perform a function as described herein. Theinstructions can be embodied in software, in firmware, or in acombination of software and firmware. The instructions can be storedlocal to the device in non-transitory media, or can be stored remotelysuch as on a server, or all or a portion of the instructions can bestored locally and remotely. Instructions stored remotely can bedownloaded (pushed) to the device by user initiation, or automaticallybased on one or more factors. It will further be appreciated that asused herein, that the terms processor, computer processor, centralprocessing unit (“CPU”), and computer are used synonymously to denote adevice capable of executing the instructions and communicating withinput/output interfaces and/or peripheral devices.

Turning now to FIG. 2, a flowchart setting forth steps of a process 200,in accordance with aspects of the present disclosure is shown. Theprocess 200 may be carried out using any suitable system, device orapparatus, such as the system 100 described with reference to FIG. 1. Insome aspects, the process 200 may be embodied in a program or softwarein the form of instructions, executable by a computer or processor, andstored in non-transitory computer-readable media.

The process 200 may begin at process block 202 with receivingperfusion-weighted imaging data. Generally speaking, theperfusion-weighted imaging data may be acquired using an MRI system, asdescribed with reference to FIG. 8, although similar acquisitions may bepossible with a CT system using different contrast agents andtechniques. For instance, the perfusion-weighted imaging data may beobtained using a perfusion acquisition such as dynamic susceptibilitycontrast (“DSC”) images or dynamic contrast enhanced (“DCE”) images thatare obtained during the administration of an intravascular contrastagent to the subject, or perfusion images gathered without the use ofcontrast agents, such as arterial spin labeling (“ASL”). Theperfusion-weighted imaging data may be obtained as a time-resolvedseries of images, in which each image in the series depicts an imagedobject or subject at a different time point. Anatomical images may alsobe received in addition to the perfusion-weighted images. As describedperfusion-weighted and other imaging data may be accessed from a memory,database, or other data storage location.

At process block 204, various selected voxels associated with thereceived perfusion-weighted imaging data may be modeled using a 4Dconvolutional neural network. In the model, one or more an input patchesmay be generated for each selected voxel. Particularly, each input patchmay be defined by a slice number s, a width w, a height h, and a volumenumber or time t, where the slice number, width and height representspatial information while the volume number represents temporalinformation. In applying the 4D convolutional neural network to thegenerated input patches, one or more spatio-temporal may be extracted,as indicated by process block 206.

By way of example, the 4D convolution neural network may include a deepCNN architecture based on multiple layers, categorized into four types:convolutional, pooling, non-linear, and fully-connected. In theconvolutional layer, each of the output neurons may be only connected toa local volume (e.g., 3 slices×5 pixels×5 pixels×40 time channels) ofthe input neurons via a weight vector, which is called feature filter(detector). Instead of common 3D feature filters (#color channel×width×height) included in typical CNNs, the convolutional layer describedherein is extended to multiple 4D feature filters (#slices× width×height×#time channels) to be learned. Through learning these 4D featuredetectors (FIG. 3), elementary features such as oriented edges,endpoints, and corners may be extracted along time from the inputpatches, and thus capturing both regional and temporal features. Eachconvolutional layer can learn multiple 4D feature filters that capturehierarchical features from the previous input layer and generate usefulfeature maps that will be used as input for the next layer. In thepooling layer, local groups of input values are “pooled” together via amax-operator to not only reduce the size of inputs, but also produceoutputs (representations) that are robust to local transformations (i.e.translation invariance).

In some aspects, a non-linear layer may be inserted between each of theconvolutional layer and pooling layer to introduce non-linearity to thenetwork. Activation functions may be used to bound neuron output. By wayof example, Rectified Linear Unit (“ReLU”) has been found to obtainconvergence in a faster manner compared to traditional activationfunctions, although other functions may be possible. The fully-connectedlayer may include output neurons that are fully connected to inputneurons (i.e. each output neuron is connected to all the input neurons).In some aspects, a deep CNN architecture may include multiple sequencesof convolutional-ReLU-pooling layers, followed by severalfully-connected layers. The last fully-connected layer can include richrepresentations that characterize a voxel input signal and thesefeatures can be used for a variety of tasks, including: 1) in a softmaxclassifier to predict tissue death, for example; 2) in a non-linear unitto estimate a perfusion parameter, for example. The weights on the 4Dconvolution neural network may be learned from batch stochastic gradientdescent via backpropagation. The hyperparameters of the network (thenumber of filters in each convolutional layer, the size of the filters,the depth of the network, and the learning rate) can be furtheroptimized via five-fold cross-validation using random search.

Referring again to FIG. 2, at process block 208, one or more perfusionparameters may then be estimated based on the extracted spatio-temporalfeatures. In some aspects, the estimated perfusion parameters may beused to generate at least one image or map. Example maps include bloodvolume (“BV”), blood flow (“BF”), mean transit time (“MTT”),time-to-peak (“TTP”), time-to-maximum (“T_(max)”), maximum signalreduction (“MSR”), first moment (“FM”) and other maps. In some aspects,various probabilities may be computed at process block 208 based on thespatio-temporal features extracted for selected voxels, using aclassifier layer in the 4D convolution neural network. By way ofexample, probabilities of infarction may be computed. As such, variousprobability maps may be generated using the computed probabilities.

A report may then be generated at process block 210. The report may bein any form, and provide various information. In some aspects, thereport may include various raw or processed maps or images, orcolor-coded maps or images. For example, the report may includeanatomical images, maps of CBF, CBV, MTT, TPP, T_(max) and otherperfusion parameters as well as probability maps. In some aspects, thereport may indicate or highlight specific regions or tissues ofinterest, as well as provide other information. The report may furtherindicate a condition of the subject or a risk of the subject todeveloping an acute or chronic condition, such as a risk of infarction.To this end, generated perfusion parameters, probabilities, maps orimages may be analyzed to determine the condition or tissue types, ortissue regions.

By way of example, FIG. 4 illustrates the steps of a process for usingCNN, in accordance with the present disclosure, to produce a probabilitymap for infarct prediction: 1) provide a patient's PWIs. For example,pre-treatment PWIs (pre-processed with skull-stripping and ventricles'removal) and PWIs acquired 3-to-7 days post-treatment usingfluid-attenuated inversion recovery (“FLAIR”) sequences may be provided;2) for a voxel in the first volume of pre-treatment PWIs, theneighboring voxels (3D volume) are selected and used to generate aninput patch with a dimension of s×w×h, where the center of the inputpatch is the voxel of interest and the dimensions s, w, h may depend onthe filters' size of the first or input layer (L1). Temporal informationmay then be included by adding another dimension to the input patch,namely t, which represents time or volumes in acquired PWIs, to producean input patch defined by s×w×h×t; As shown in FIG. 4, as a non-limitingexample, an input patch may be 1×21×21×40, although other dimensions maybe possible; 3) the CNN consisting of a convolutional layer (L2), apooling layer (L3), a first fully connected layer (L4) and a secondfully connected layer (L5) may then be applied to the input patch andfeatures may generated in the second fully-connected layer (L5); For theoutput layer (L6), a softmax classifier layer may be used to calculatethe probability of infarction based on the features; Steps 1 and 2 maybe repeated by feeding each input patch into the network, until alldesired voxels are analyzed, thereby producing a number ofprobabilities, that can be assembled to produce one or more infarctionprobability maps, as shown in FIG. 4. In some alternatives, the abovesteps may be repeated to produce perfusion parameters, and one or moreperfusion maps therefrom (e.g., T_(max) maps). To this end, a non-linearlayer may be used instead of the softmax classifier layer in the outputlayer (L6).

In demonstrating the present approach, two deep CNNs were built inaccordance with the present disclosure to perform infarction predictionand T_(max) estimation, with results shown in FIG. 5A-5B. FIG. 5A showsa probability map (bottom) generated in accordance with the presentdisclosure in comparison to a post-treatment FLAIR image (top). Regions500 with a high probability of infarction (dark red) are mostly alignedwith true infarcted tissue as measured on the FLAIR image acquired 3-7days after treatment, illustrating that the deep CNN approach is able tocorrectly predict infarctions. FIG. 5B shows a T_(max) perfusion map(bottom) estimated in accordance with the present disclosure incomparison to a labeled perfusion map (top). It may be observed that theestimated perfusion map is mostly aligned with the labeled perfusionmap, showing effectiveness of the method. In additional, the resultsshow that the present deep CNN is capable of learning spatio-temporalfilters for PWIs. As an illustration, FIG. 6 shows two examplespatio-temporal filters from pre-treatment PWIs. Each row corresponds toone filter, and each column corresponds to a 2D filter at a particularvolume (time) in pre-treatment PWIs. The spatial changes of theintensity along volumes (times) of a filter demonstrate the ability ofthe present approach to extract spatial and temporal information fromthe inputs. As such, these powerful filters extract hierarchicalfeatures can be useful for the prediction of infarcts.

Referring particularly now to FIG. 7, an example of a magnetic resonanceimaging (“MRI”) system 700 that may be used to implement the systems andmethods of the present disclosure is illustrated. The MRI system 700includes a workstation 702 having a display 704 and a keyboard 706. Theworkstation 702 includes a processor 708, such as a commerciallyavailable programmable machine running a commercially availableoperating system. The workstation 702 provides the operator interfacethat enables scan prescriptions to be entered into the MRI system 700.The workstation 702 is coupled to servers, including a pulse sequenceserver 710, a data acquisition server 712, a data processing server 714,and a data store server 716. The workstation 702 and each server 710,712, 714, and 716 are in communication.

The pulse sequence server 710 functions in response to instructionsdownloaded from the workstation 702 to operate a gradient system 718 anda radiofrequency (“RF”) system 720. Gradient waveforms necessary toperform the prescribed scan are produced and applied to the gradientsystem 718, which excites gradient coils in an assembly 722 to producethe magnetic field gradients G_(x), G_(y), and G_(z) used for positionencoding MR signals. The gradient coil assembly 722 forms part of amagnet assembly 724 that includes a polarizing magnet 726 and awhole-body RF coil 728.

RF excitation waveforms are applied to the RF coil 728, or a separatelocal coil (not shown in FIG. 7), by the RF system 720 to perform theprescribed magnetic resonance pulse sequence. Responsive MR signalsdetected by the RF coil 728, or a separate local coil, are received bythe RF system 720, amplified, demodulated, filtered, and digitized underdirection of commands produced by the pulse sequence server 710. The RFsystem 720 includes an RF transmitter for producing a wide variety of RFpulses used in MR pulse sequences. The RF transmitter is responsive tothe scan prescription and direction from the pulse sequence server 710to produce RF pulses of the desired frequency, phase, and pulseamplitude waveform. The generated RF pulses may be applied to the wholebody RF coil 728 or to one or more local coils or coil arrays.

The RF system 720 also includes one or more RF receiver channels. EachRF receiver channel includes an RF preamplifier that amplifies the MRsignal received by the coil 728 to which it is connected, and a detectorthat detects and digitizes the I and Q quadrature components of thereceived MR signal. The magnitude of the received MR signal may thus bedetermined at any sampled point by the square root of the sum of thesquares of the I and Q components:M=√{square root over (I ² +Q ²)}  (1);

and the phase of the received MR signal may also be determined:

$\begin{matrix}{\varphi = {{\tan^{- 1}\left( \frac{Q}{I} \right)}.}} & (2)\end{matrix}$

The pulse sequence server 710 also optionally receives patient data froma physiological acquisition controller 730. The controller 730 receivessignals from a number of different sensors connected to the patient,such as electrocardiograph (“ECG”) signals from electrodes, orrespiratory signals from a bellows or other respiratory monitoringdevice. Such signals are typically used by the pulse sequence server 710to synchronize, or “gate,” the performance of the scan with thesubject's heart beat or respiration.

The pulse sequence server 710 also connects to a scan room interfacecircuit 732 that receives signals from various sensors associated withthe condition of the patient and the magnet system. It is also throughthe scan room interface circuit 732 that a patient positioning system734 receives commands to move the patient to desired positions duringthe scan.

The digitized MR signal samples produced by the RF system 720 arereceived by the data acquisition server 712. The data acquisition server712 operates in response to instructions downloaded from the workstation702 to receive the real-time MR data and provide buffer storage, suchthat no data is lost by data overrun. In some scans, the dataacquisition server 712 does little more than pass the acquired MR datato the data processor server 714. However, in scans that requireinformation derived from acquired MR data to control the furtherperformance of the scan, the data acquisition server 712 is programmedto produce such information and convey it to the pulse sequence server710. For example, during prescans, MR data is acquired and used tocalibrate the pulse sequence performed by the pulse sequence server 710.Also, navigator signals may be acquired during a scan and used to adjustthe operating parameters of the RF system 720 or the gradient system718, or to control the view order in which k-space is sampled. In allthese examples, the data acquisition server 712 acquires MR data andprocesses it in real-time to produce information that is used to controlthe scan.

The data processing server 714 receives MR data from the dataacquisition server 712 and processes it in accordance with instructionsdownloaded from the workstation 702. Such processing may include, forexample: Fourier transformation of raw k-space MR data to produce two orthree-dimensional images; the application of filters to a reconstructedimage; the performance of a backprojection image reconstruction ofacquired MR data; the generation of functional MR images; and thecalculation of motion or flow images.

Images reconstructed by the data processing server 714 are conveyed backto the workstation 702 where they are stored. Real-time images arestored in a data base memory cache, from which they may be output tooperator display 712 or a display 736 that is located near the magnetassembly 724 for use by attending physicians. Batch mode images orselected real time images are stored in a host database on disc storage738. When such images have been reconstructed and transferred tostorage, the data processing server 714 notifies the data store server716 on the workstation 702.

The MRI system 700 may also include one or more networked workstations742. By way of example, a networked workstation 742 may include adisplay 744, one or more input devices 746 (such as a keyboard and mouseor the like), and a processor 748. The networked workstation 742 may belocated within the same facility as the operator workstation 702, or ina different facility, such as a different healthcare institution orclinic. The networked workstation 742 may include a mobile device,including phones or tablets.

The networked workstation 742, whether within the same facility or in adifferent facility as the operator workstation 702, may gain remoteaccess to the data processing server 714 or data store server 716 viathe communication system 740. Accordingly, multiple networkedworkstations 742 may have access to the data processing server 714 andthe data store server 716. In this manner, magnetic resonance data,reconstructed images, or other data may exchanged between the dataprocessing server 714 or the data store server 716 and the networkedworkstations 742, such that the data or images may be remotely processedby a networked workstation 742. This data may be exchanged in anysuitable format, such as in accordance with the transmission controlprotocol (“TCP”), the internet protocol (“IP”), or other known orsuitable protocols.

The present invention has been described in terms of one or morepreferred embodiments, and it should be appreciated that manyequivalents, alternatives, variations, and modifications, aside fromthose expressly stated, are possible and within the scope of theinvention.

The invention claimed is:
 1. A computer-implemented method for analyzingperfusion-weighted imaging, the method comprising: a) receivingperfusion-weighted imaging data acquired from a subject having receivedan injection of a contrast agent using a magnetic resonance (“MR”)imaging system; b) modeling at least one voxel associated with theperfusion-weighted imaging data using a four-dimensional (“4D”)convolutional neural network; c) extracting spatio-temporal features foreach modeled voxel; d) estimating at least one perfusion parameter foreach modeled voxel based on the extracted spatio-temporal features; ande) generating a report using the at least one perfusion parameterindicating perfusion in the subject.
 2. The method of claim 1, whereinthe perfusion-weighted imaging data comprises at least one of dynamicsusceptibility contrast (“DSC”) imaging data and dynamic contrastenhanced (“DCE”) imaging data.
 3. The method of claim 1, wherein the 4Dconvolutional neural network comprises spatial information and temporalinformation.
 4. The method of claim 1, wherein the 4D convolution neuralnetwork comprises at least one of a convolutional, a pooling, anon-linear, and a fully-connected layer.
 5. The method of claim 4,wherein the 4D convolution neural network further comprises a non-linearlayer inserted between each of the convolutional layer and the poolinglayer.
 6. The method of claim 1, wherein the method further comprisesgenerating at least one input patch defined by a slice number, a width,a height, and a volume number for each modeled voxel.
 7. The method ofclaim 6, wherein the method further comprises providing the at least oneinput patch to the 4D convolutional neural network to extract thespatio-temporal features.
 8. The method of claim 1, wherein the methodfurther comprises computing, using a classifier layer in the 4Dconvolution neural network, at least one probability of infarction basedon the spatio-temporal features extracted.
 9. The method of claim 8,wherein the method further comprises generating a probability map usingthe at least one probability computed.
 10. The method of claim 1,wherein the method further comprises determining a condition of thesubject based on the at least one perfusion parameter.
 11. The method ofclaim 1, wherein the at least one perfusion parameter comprises at leastone of a blood volume (“BV”), a blood flow (“BF”), a mean transit time(“MTT”), a time-to-peak (“TTP”), a time-to-maximum (“T_(max)”), amaximum signal reduction (“MSR”), and a first moment (“FM”).
 12. Asystem for analyzing perfusion-weighted imaging, the system comprising:an input configured to receive perfusion-weighted imaging data acquiredfrom a subject using a magnetic resonance (“MR”) imaging system; aprocessor programmed execute instructions stored in a non-transitorycomputer readable media to: i) access the perfusion-weighted imagingdata; ii) generate an input patch for each selected voxel associatedwith the perfusion-weighed imaging data; iii) apply a four-dimensional(“4D”) convolutional neural network to the generated input patch toextract spatio-temporal features; iv) compute, at least one perfusionparameter using the spatio-temporal features extracted; v) generate areport using the at least one perfusion parameter; and an output fordisplaying the report to communicate perfusion in the subject.
 13. Thesystem of claim 12, wherein the perfusion-weighted imaging datacomprises at least one of dynamic susceptibility contrast (“DSC”)imaging data and dynamic contrast enhanced (“DCE”) imaging data.
 14. Thesystem of claim 12, wherein the 4D convolutional neural networkcomprises spatial information and temporal information.
 15. The systemof claim 12, wherein the 4D convolution neural network comprises atleast one of a convolutional, a pooling, a non-linear, and afully-connected layer.
 16. The system of claim 15, wherein the 4Dconvolution neural network further comprises a non-linear layer isinserted between each of the convolutional layer and the pooling layer.17. The system of claim 12, wherein the processor is further programmedto compute, using a classifier layer in the 4D convolution neuralnetwork, at least one probability of infarction based on thespatio-temporal features extracted.
 18. The system of claim 17, whereinthe processor is further programmed to generate a probability map usingthe at least one probability computed for each of plurality of selectedvoxels.
 19. The system of claim 12, wherein the at least one perfusionparameter comprises at least one of a blood volume (“BV”), a blood flow(“BF”), a mean transit time (“MTT”), a time-to-peak (“TTP”), atime-to-maximum (“T_(max)”), a maximum signal reduction (“MSR”), and afirst moment (“FM”).
 20. The system of claim 12, wherein the processoris further programmed to generate at least one image by the at least oneperfusion parameter computed for each of a plurality of selected voxels.21. The system of claim 11, wherein the processor is further programmedto determine a condition of the subject based on the at least oneperfusion parameter.
 22. The method of claim 1, wherein the 4Dconvolutional neural network is configured to capture spatial for eachmodeled voxel in three dimensions (“3D”) and capture temporalinformation of each modeled voxel in a fourth dimension of the 4Dconvolutional neural network.
 23. The system of claim 12, wherein the 4Dconvolutional neural network is configured to capture spatial for eachmodeled voxel in three dimensions (“3D”) and capture temporalinformation of each modeled voxel in a fourth dimension of the 4Dconvolutional neural network.